{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q0FyHWcc6RI9"
      },
      "source": [
        "## Article Recommender with Text Embedding, Classification, and Extraction"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nGX0RfPUfFgq"
      },
      "source": [
        "This is a simple demonstration of how we can stack multiple NLP models together \n",
        "to get an output much closer to our desired outcome."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VkGvDUYd2gnh"
      },
      "source": [
        "Embeddings can capture the meaning of a piece of text beyond keyword-matching. In this article, we will build a simple news article recommender system that computes the embeddings of all available articles and recommend the most relevant articles based on embeddings similarity. \n",
        "\n",
        "We will also make the recommendation tighter by using text classification to recommend only articles within the same category. We will then extract a list of tags from each recommended article, which can further help readers discover new articles. \n",
        "\n",
        "All this will be done via three Cohere API endpoints stacked together: Embed, Classify, and Generate.\n",
        "\n",
        "![Article recommender with Embed, Classify, and Generate](https://github.com/cohere-ai/notebooks/raw/main/notebooks/images/article-recommender/article-rec-1.png)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fNvRai38Isqr"
      },
      "source": [
        "We will implement the following steps:\n",
        "\n",
        "**1: Find the most similar articles to the one currently reading using embeddings.**\n",
        "\n",
        "**2: Keep only articles of the same category using text classification.**\n",
        "\n",
        "**3: Extract tags from these articles.**\n",
        "\n",
        "**4: Show the top 5 recommended articles.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "29uwe-jzJ9rh",
        "outputId": "d85ab1db-36d4-4fc3-bbba-319b88ba158f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting cohere\n",
            "  Downloading cohere-1.3.10-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.0 MB)\n",
            "\u001b[K     |████████████████████████████████| 18.0 MB 135 kB/s \n",
            "\u001b[?25hRequirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from cohere) (2.23.0)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->cohere) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->cohere) (2022.6.15)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->cohere) (2.10)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->cohere) (3.0.4)\n",
            "Installing collected packages: cohere\n",
            "Successfully installed cohere-1.3.10\n"
          ]
        }
      ],
      "source": [
        "# TODO: upgrade to \"cohere>5\"",
"! pip install \"cohere<5\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 68,
      "metadata": {
        "id": "y9-RyLu7KHII"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import re\n",
        "import cohere\n",
        "api_key = 'COHERE_API_KEY' # Paste your API key here. Remember to not share it publicly \n",
        "co = cohere.Client(api_key)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SeR8yeXl4YtQ"
      },
      "source": [
        "# **1: Find the most similar articles to the one currently reading using embeddings**\n",
        "\n",
        "![Step 1 - Embed](https://github.com/cohere-ai/notebooks/raw/main/notebooks/images/article-recommender/article-rec-2.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Efe2XFpWqsw7"
      },
      "source": [
        "## 1.1: Get articles"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MJMuR_Pb5TUh"
      },
      "source": [
        "Throughout this article, we'll use the [BBC news article dataset](https://www.kaggle.com/competitions/learn-ai-bbc/data?select=BBC+News+Train.csv) as an example [[Source]](http://mlg.ucd.ie/datasets/bbc.html). This dataset consists of articles from a few categories: business, politics, tech, entertainment, and sport.\n",
        "\n",
        "We'll extract a subset of the data and in Step 1, use the first 100 data points.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "G95txfh1KHCu",
        "outputId": "3b58f535-d1b9-4862-f6d1-e2aa5e37495c"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>worldcom ex-boss launches defence lawyers defe...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>german business confidence slides german busin...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>bbc poll indicates economic gloom citizens in ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>lifestyle  governs mobile choice  faster  bett...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>enron bosses in $168m payout eighteen former e...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                Text\n",
              "0  worldcom ex-boss launches defence lawyers defe...\n",
              "1  german business confidence slides german busin...\n",
              "2  bbc poll indicates economic gloom citizens in ...\n",
              "3  lifestyle  governs mobile choice  faster  bett...\n",
              "4  enron bosses in $168m payout eighteen former e..."
            ]
          },
          "execution_count": 2,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Load the dataset to a dataframe\n",
        "df = pd.read_csv('https://raw.githubusercontent.com/cohere-ai/notebooks/main/notebooks/data/bbc_news_subset.csv', delimiter=',')\n",
        "\n",
        "# Select a portion of the dataset \n",
        "INP_START = 0\n",
        "INP_END = 100\n",
        "df_inputs = df.iloc[INP_START:INP_END]\n",
        "df_inputs = df_inputs.copy()\n",
        "\n",
        "# Remove columns we don't need\n",
        "df_inputs.drop(['ArticleId','Category'],axis=1,inplace=True)\n",
        "\n",
        "# View the data\n",
        "df_inputs.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O_kIeQpENJzp"
      },
      "source": [
        "## 1.2: Turn articles into embeddings\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wu3fRvs3A5oe"
      },
      "source": [
        "Next we turn each article text into embeddings. An [embedding](https://docs.cohere.ai/embedding-wiki) is a list of numbers that our models use to represent a piece of text, capturing its context and meaning.\n",
        "\n",
        "We do this by calling Cohere's [Embed endpoint](https://docs.cohere.ai/embed-reference), which takes in text as input and returns embeddings as output."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CoLcyJLsKHAN",
        "outputId": "d3e53c90-9505-4a5d-ef65-135312b7b0eb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Number of articles: 100\n"
          ]
        }
      ],
      "source": [
        "# Get text embeddings via the Embed endpoint\n",
        "articles = df_inputs['Text'].tolist()\n",
        "\n",
        "output = co.embed(\n",
        "            model ='embed-english-v3.0',\n",
        "            input_type='search_document',\n",
        "            texts = articles)\n",
        "embeds = output.embeddings\n",
        "\n",
        "print('Number of articles:', len(embeds))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UqOIWI_LM2Rc"
      },
      "source": [
        "## 1.3: Pick one article and find the most similar articles"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MIKvKKgJBcnK"
      },
      "source": [
        "Next, we pick any one article to be the one the reader is currently reading (let's call this the target) and find other articles with the most similar embeddings (let's call these candidates) using cosine similarity.\n",
        "\n",
        "[Cosine similarity](https://en.wikipedia.org/wiki/Cosine_similarity) is a metric that measures how similar sequences of numbers are (embeddings in our case), and we compute it for each target-candidate pair. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qfJJZ6Z0NaWg",
        "outputId": "40baea0c-9d1e-4fa2-a75e-3b0f0ba491ce"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Choose one article ID between 0 and 99 below...\n"
          ]
        }
      ],
      "source": [
        "# Choose one article ID as the one you are currently reading\n",
        "print(f'Choose one article ID between {INP_START} and {INP_END-1} below...')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "yn0lvlBuNwq_"
      },
      "outputs": [],
      "source": [
        "# Enter your article ID\n",
        "READING_IDX = 70\n",
        "\n",
        "# Get embedding for the article\n",
        "reading = embeds[READING_IDX]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "qhhaiIsEQF9k"
      },
      "outputs": [],
      "source": [
        "# Calculate cosine similarity between the target and candidate articles\n",
        "\n",
        "from sklearn.metrics.pairwise import cosine_similarity\n",
        "\n",
        "def get_similarity(target,candidates):\n",
        "  # Turn list into array\n",
        "  candidates = np.array(candidates)\n",
        "  target = np.expand_dims(np.array(target),axis=0)\n",
        "\n",
        "  # Calculate cosine similarity\n",
        "  similarity_scores = cosine_similarity(target,candidates)\n",
        "  similarity_scores = np.squeeze(similarity_scores).tolist()\n",
        "\n",
        "  # Sort by descending order in similarity\n",
        "  similarity_scores = list(enumerate(similarity_scores))\n",
        "  similarity_scores = sorted(similarity_scores, key=lambda x:x[1], reverse=True)\n",
        "\n",
        "  # Return similarity scores\n",
        "  return similarity_scores"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jVNQ0cUDR-5U",
        "outputId": "040e57ad-ff14-4c19-97f1-554edc4dac59"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Target:\n",
            "[ID 70] aragones angered by racism fine spain coach luis aragones is furious after being fined by the spanis ... \n",
            "\n",
            "Candidates:\n",
            "[ID 23] ferguson urges henry punishment sir alex ferguson has called on the football association to punish a ...\n",
            "[ID 51] mourinho defiant on chelsea form chelsea boss jose mourinho has insisted that sir alex ferguson and  ...\n",
            "[ID 73] balco case trial date pushed back the trial date for the bay area laboratory cooperative (balco) ste ...\n",
            "[ID 41] mcleish ready for criticism rangers manager alex mcleish accepts he is going to be criticised after  ...\n",
            "[ID 42] premier league planning cole date the premier league is attempting to find a mutually convenient dat ...\n"
          ]
        }
      ],
      "source": [
        "# Get the similarity between the target and candidate articles\n",
        "similarity = get_similarity(reading,embeds)\n",
        "\n",
        "# View the top 5 articles\n",
        "print('Target:')\n",
        "print(f'[ID {READING_IDX}]',df_inputs['Text'][READING_IDX][:100],'...','\\n')\n",
        "\n",
        "print('Candidates:')\n",
        "for i in similarity[1:6]: # Exclude the target article\n",
        "  print(f'[ID {i[0]}]',df_inputs['Text'][i[0]][:100],'...')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZpeETEhfyvOa"
      },
      "source": [
        "# **2: Keep only articles of the same category using text classification**\n",
        "\n",
        "![Step 2 - Classify](https://github.com/cohere-ai/notebooks/raw/main/notebooks/images/article-recommender/article-rec-3.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9-neV3KEyyQP"
      },
      "source": [
        "In the example above (Article ID 70 as the target), we see that the top 5 most similar articles given by the system are very relevant. The target is a football/soccer article, and the system duly recommended very similar articles despite this dataset also containing articles from other sports like tennis and rugby.\n",
        " \n",
        "However, not all of them are. The fourth recommended article (ID 86) is not a sports article, but rather politics. Reading the text, it's likely because the target is news about a clash of individuals (i.e. anger about a racism fine), which happens to be what that politics article is also about (i.e. disagreement over an apology). So these two articles' meanings are similar in this way, captured in the embeddings.\n",
        " \n",
        "Perhaps we can make the system better by only recommending articles of the same category. For this, let's build a news category classifier."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "viuvclW_fqdf"
      },
      "source": [
        "## 2.1: Build a classifier\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7GFksI1A8mHS"
      },
      "source": [
        "We use Cohere's [Classify endpoint](https://docs.cohere.ai/classify-reference) to build a news category classifier, classifying articles into five categories: Business, Politics, Tech, Entertainment, and Sport. \n",
        "\n",
        "A typical text classification model requires hundreds/thousands of data points to train, but with this endpoint, we can build a classifier with a few as five examples per class.\n",
        "\n",
        "To build the classifier, we need a set of examples consisting of text (news text) and labels (news category). The BBC News dataset happens to have both (columns 'Text' and 'Category'), so this time we’ll use the categories for building our examples. For this, we will set aside another portion of dataset."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "pBJ5SQ5-zrJr",
        "outputId": "84a72cd9-4a8b-491e-c8da-4a8f46da4a99"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Text</th>\n",
              "      <th>Category</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>100</th>\n",
              "      <td>honda wins china copyright ruling japan s hond...</td>\n",
              "      <td>business</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>101</th>\n",
              "      <td>ukip could sue veritas defectors the uk indepe...</td>\n",
              "      <td>politics</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>102</th>\n",
              "      <td>security warning over  fbi virus  the us feder...</td>\n",
              "      <td>tech</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>103</th>\n",
              "      <td>europe backs digital tv lifestyle how people r...</td>\n",
              "      <td>tech</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>104</th>\n",
              "      <td>celebrities get to stay in jungle all four con...</td>\n",
              "      <td>entertainment</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                  Text       Category\n",
              "100  honda wins china copyright ruling japan s hond...       business\n",
              "101  ukip could sue veritas defectors the uk indepe...       politics\n",
              "102  security warning over  fbi virus  the us feder...           tech\n",
              "103  europe backs digital tv lifestyle how people r...           tech\n",
              "104  celebrities get to stay in jungle all four con...  entertainment"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Select a portion of the dataset to sample the classification examples from\n",
        "EX_START = 100\n",
        "EX_END = 200\n",
        "df_examples = df.iloc[EX_START:EX_END]\n",
        "df_examples = df_examples.copy()\n",
        "\n",
        "# Remove columns we don't need\n",
        "df_examples.drop(['ArticleId'],axis=1,inplace=True)\n",
        "\n",
        "# View the data\n",
        "df_examples.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4e2_yP0t-ciE"
      },
      "source": [
        "With the Classify endpoint, there is a limit of 2048 tokens with the medium model. This means full articles won't be able to fit in the examples, so we will approximate and limit each article to its first 300 characters."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "Z7l_B_87-dfT"
      },
      "outputs": [],
      "source": [
        "# Shorten the example articles (because the medium endpoint max token limit is 2048)\n",
        "MAX_CHARS = 300\n",
        "\n",
        "def shorten_text(text):\n",
        "  return text[:MAX_CHARS]\n",
        "\n",
        "df_examples['Text'] = df_examples['Text'].apply(shorten_text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hkqfMw019fvH"
      },
      "source": [
        "The Classify endpoint needs a minimum of 2 examples for each category. We'll have 5 examples each, sampled randomly from the dataset. We have 5 categories, so we will have a total of 25 examples."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OUdZ6y3g-ZEN",
        "outputId": "69076931-fa30-4602-cbb0-f9f4b45dc86f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Number of examples per category: 5\n",
            "List of categories: ['business', 'politics', 'tech', 'entertainment', 'sport']\n",
            "Number of categories: 5\n",
            "Total number of examples: 25\n"
          ]
        }
      ],
      "source": [
        "# Set the number of examples per category\n",
        "EX_PER_CAT = 5 \n",
        "\n",
        "# Get the list of all available categories\n",
        "categories = df_examples['Category'].unique().tolist()\n",
        "\n",
        "# Create list of examples containing texts and labels\n",
        "ex_texts = []\n",
        "ex_labels = []\n",
        "for category in categories:\n",
        "  df_category = df_examples[df_examples['Category'] == category]\n",
        "  samples = df_category.sample(n=EX_PER_CAT, random_state=42)\n",
        "  ex_texts += samples['Text'].tolist()\n",
        "  ex_labels += samples['Category'].tolist()\n",
        "\n",
        "print(f'Number of examples per category: {EX_PER_CAT}')\n",
        "print(f'List of categories: {categories}')\n",
        "print(f'Number of categories: {len(categories)}')\n",
        "print(f'Total number of examples: {len(ex_texts)}')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZuFzl0IS--SA"
      },
      "source": [
        "Once the examples are ready, we can now get the classifications. Here is a function that returns the classification given an input."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "HNFPu99N_Agc"
      },
      "outputs": [],
      "source": [
        "# Get classifications via the Classify endpoint\n",
        "\n",
        "from cohere.responses.classify import Example\n",
        "\n",
        "# Collate the examples\n",
        "examples = []\n",
        "for txt, lbl in zip(ex_texts,ex_labels):\n",
        "  examples.append(Example(txt,lbl))\n",
        "\n",
        "# Classification function\n",
        "def classify_text(texts, examples):\n",
        "    classifications = co.classify(\n",
        "        inputs=texts,\n",
        "        examples=examples\n",
        "    )\n",
        "    return [c.predictions[0] for c in classifications]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mQ3HJ8Ep4uYk"
      },
      "source": [
        "## 2.2: Measure its performance"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "spu8JHGX_C68"
      },
      "source": [
        "Before actually using the classifier, let's first test its performance. Here we take another 100 data points as the test dataset and the classifier will predict its class i.e. news category."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "-OwI7MzZ0Ev0",
        "outputId": "66d1ccf1-98db-4962-b313-3548cba5649b"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Text</th>\n",
              "      <th>Category</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>200</th>\n",
              "      <td>sa return to mauritius top seeds south africa ...</td>\n",
              "      <td>sport</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>201</th>\n",
              "      <td>snow patrol feted at irish awards snow patrol ...</td>\n",
              "      <td>entertainment</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>202</th>\n",
              "      <td>clyde 0-5 celtic celtic brushed aside clyde to...</td>\n",
              "      <td>sport</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>203</th>\n",
              "      <td>bad weather hits nestle sales a combination of...</td>\n",
              "      <td>business</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>204</th>\n",
              "      <td>net fingerprints combat attacks eighty large n...</td>\n",
              "      <td>tech</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                  Text       Category\n",
              "200  sa return to mauritius top seeds south africa ...          sport\n",
              "201  snow patrol feted at irish awards snow patrol ...  entertainment\n",
              "202  clyde 0-5 celtic celtic brushed aside clyde to...          sport\n",
              "203  bad weather hits nestle sales a combination of...       business\n",
              "204  net fingerprints combat attacks eighty large n...           tech"
            ]
          },
          "execution_count": 21,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Select a portion of the dataset for testing the classifier\n",
        "TEST_START = 200\n",
        "TEST_END = 300\n",
        "df_test = df.iloc[TEST_START:TEST_END]\n",
        "df_test = df_test.copy()\n",
        "\n",
        "# Remove columns we don't need\n",
        "df_test.drop(['ArticleId'],axis=1,inplace=True)\n",
        "\n",
        "# Shorten the text to fit token limit\n",
        "df_test['Text'] = df_test['Text'].apply(shorten_text)\n",
        "\n",
        "# View the data\n",
        "df_test.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "Kkt3E5_c_O5c"
      },
      "outputs": [],
      "source": [
        "# Create batches of texts and classify them\n",
        "predictions = []\n",
        "BATCH_SIZE = 90 # The API accepts a maximum of 96 inputs\n",
        "for i in range(0, len(df_test['Text']), BATCH_SIZE):\n",
        "    batch_texts = df_test['Text'][i:i+BATCH_SIZE].tolist()\n",
        "    predictions.extend(classify_text(batch_texts, examples))\n",
        "    \n",
        "    \n",
        "# Actual classes\n",
        "actual = df_test['Category'].tolist()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nZ10tnuB_QKf",
        "outputId": "f3dc05c5-6dc3-456f-dc17-ca81fd69e323"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Accuracy: 89.0\n"
          ]
        }
      ],
      "source": [
        "# Compute metrics on the test dataset\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "accuracy = accuracy_score(actual, predictions)\n",
        "print(f'Accuracy: {accuracy*100}')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F0Rqzz1w_SiB"
      },
      "source": [
        "We get a good accuracy score of 91%, so the classifier is ready to be \n",
        "implemented in our recommender system."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MaY7t_1DJBiy"
      },
      "source": [
        "# **3: Extract tags from these articles.**\n",
        "\n",
        "![Step 3 - Extract](https://github.com/cohere-ai/notebooks/raw/main/notebooks/images/article-recommender/article-rec-4.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x8hoJCh7Tv01"
      },
      "source": [
        "We now proceed to the tags extraction step. Compared to the previous two steps, this step is not about sorting or filtering articles, but rather enriching them with more information. \n",
        "\n",
        "We do this by [prompting](https://docs.cohere.ai/prompt-engineering-wiki) Cohere's [Generate endpoint](https://docs.cohere.ai/generate-reference) with a few examples of text and its tags. We then feed the articles from the classifier step and the endpoint will generate the corresponding tags.\n",
        "\n",
        "There are more than one way to construct the prompt, depending on what you'd lile to extract. In my case, the tags I'd like to extract are primarily the names of a person, company, or organization, and perhaps also some generic keywords. That was the idea behind the example tags I put in the prompt, which you can see on the Cohere Playground screenshot below...\n",
        "\n",
        "![Tag extraction prompt](https://github.com/cohere-ai/notebooks/raw/main/notebooks/images/article-recommender/article-rec-6.png)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 46,
      "metadata": {
        "id": "tIhfHWO0JBRd"
      },
      "outputs": [],
      "source": [
        "# Create the base prompt containing extraction examples\n",
        "# base_prompt = 'Given a news article, please return the list tags containing proper nouns.\\n\\nArticle: japanese banking battle at an end japan s sumitomo mitsui financial has withdrawn its takeover offer for rival bank ufj holdings  enabling the latter to merge with mitsubishi tokyo.  sumitomo bosses told counterparts at ufj of its decision on friday  clearing the way for it to conclude a 3 trillion\\n\\nTags: sumitomo mitsui financial, ufj holdings, mitsubishi tokyo\\n--\\nArticle: france starts digital terrestrial france has become the last big european country to launch a digital terrestrial tv (dtt) service.  initially  more than a third of the population will be able to receive 14 free-to-air channels. despite the long wait for a french dtt roll-out  the new platform s bac\\n\\nTags: france\\n--\\nArticle: apple laptop is  greatest gadget  the apple powerbook 100 has been chosen as the greatest gadget of all time  by us magazine mobile pc.  the 1991 laptop was chosen because it was one of the first  lightweight  portable computers and helped define the layout of all future notebook pcs. the magazine h\\n\\nTags: apple, apple powerbook 100, mobile pc\\n--\\nArticle:'\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yqg40uJbXs9T"
      },
      "source": [
        "We call the endpoint by specifying a few settings, and it will generate the corresponding extractions."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 61,
      "metadata": {
        "id": "6j0t4VjJJBNp"
      },
      "outputs": [],
      "source": [
        "# Get extractions via the Generate endpoint\n",
        "def extract_tags(article):\n",
        "  prompt = f\"\"\"Given an article, extract a list of tags containing keywords \\\n",
        "of that article.\\n\\nArticle: japanese banking battle at an end japan s sumitomo mitsui \\\n",
        "financial has withdrawn its takeover offer for rival bank ufj holdings  enabling the \\\n",
        "latter to merge with mitsubishi tokyo.  sumitomo bosses told counterparts at ufj of its \\\n",
        "decision on friday  clearing the way for it to conclude a 3 trillion\\n\\nTags: \\\n",
        "sumitomo mitsui financial, ufj holdings, mitsubishi tokyo, japanese banking\\n--\\nArticle: \\\n",
        "france starts digital terrestrial france has become the last big european country to \\\n",
        "launch a digital terrestrial tv (dtt) service.  initially  more than a third of the \\\n",
        "population will be able to receive 14 free-to-air channels. despite the long wait for a \\\n",
        "french dtt roll-out  the new platform s bac\\n\\nTags: france, digital terrestrial\\n--\\nArticle: \\\n",
        "apple laptop is  greatest gadget  the apple powerbook 100 has been chosen as the greatest \\\n",
        "gadget of all time  by us magazine mobile pc.  the 1991 laptop was chosen because it was \\\n",
        "one of the first  lightweight  portable computers and helped define the layout of all future \\\n",
        "notebook pcs. the magazine h\\n\\nTags: apple, apple powerbook 100, laptop\\n--\\nArticle:{article}\\n\\nTags:\"\"\"\n",
        "  \n",
        "  \n",
        "  prediction = co.generate(\n",
        "    model='command',\n",
        "    prompt=prompt,\n",
        "    max_tokens=50,\n",
        "    temperature=0.3)\n",
        "\n",
        "  return prediction.generations[0].text"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fzcXdkhR6hVy"
      },
      "source": [
        "# **4: Show the top 5 recommended articles.**\n",
        "\n",
        "![Complete all steps](https://github.com/cohere-ai/notebooks/raw/main/notebooks/images/article-recommender/article-rec-5.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mLyUW3Gp4KRB"
      },
      "source": [
        "Let's now put everything together for our article recommender system.\n",
        "\n",
        "First, we select the target article and compute the similarity scores against the candidate articles."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 62,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XVTSzK6AXoS3",
        "outputId": "f1f690fb-d281-41f5-abb3-8eafa4c5a172"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Choose one article ID between 0 and 99 below...\n"
          ]
        }
      ],
      "source": [
        "# Choose one article ID as the one you are currently reading\n",
        "print(f'Choose one article ID between {INP_START} and {INP_END-1} below...')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 63,
      "metadata": {
        "id": "mM7mBSGvJBKd"
      },
      "outputs": [],
      "source": [
        "# Enter your article ID\n",
        "READING_IDX = 70\n",
        "\n",
        "# Get embedding for the article\n",
        "reading = embeds[READING_IDX]\n",
        "\n",
        "# Get the similarity between the target and candidate articles\n",
        "similarity = get_similarity(reading,embeds)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ECFlqp2k5jET"
      },
      "source": [
        "Next, we filter the articles via classification. Finally, we extract the keywords from each article and show the recommendations.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 64,
      "metadata": {
        "id": "eviOxPUZJBGo"
      },
      "outputs": [],
      "source": [
        "# Choose the number of articles to recommend\n",
        "SHOW_TOP = 5\n",
        "\n",
        "# Shorten the text to fit token limit\n",
        "df_inputs = df_inputs.copy()\n",
        "df_inputs['Text'] = df_inputs['Text'].apply(shorten_text)\n",
        "\n",
        "# Get the recommendations\n",
        "def get_recommendations(reading_idx,similarity,show_top):\n",
        "\n",
        "  # Show the current article\n",
        "  print('------  You are reading...  ------')\n",
        "  print(f'[ID {READING_IDX}] Article:',df_inputs['Text'][reading_idx][:MAX_CHARS]+'...\\n')\n",
        "\n",
        "  # Show the recommended articles\n",
        "  print('------  You might also like...  ------')\n",
        "\n",
        "  # Classify the target article\n",
        "  target_class = classify_text([df_inputs['Text'][reading_idx]],examples) \n",
        "\n",
        "  count = 0\n",
        "  for idx,score in similarity:\n",
        "\n",
        "    # Classify each candidate article\n",
        "    candidate_class = classify_text([df_inputs['Text'][idx]],examples)\n",
        "\n",
        "    # Show recommendations\n",
        "    if target_class == candidate_class and idx != reading_idx:\n",
        "      selection = df_inputs['Text'][idx][:MAX_CHARS]\n",
        "      print(f'[ID {idx}] Article:',selection+'...')\n",
        "\n",
        "      # Extract and show tags\n",
        "      tags = extract_tags(selection)\n",
        "      if tags:\n",
        "          print(f'Tags: {tags.strip()}\\n')\n",
        "      else:\n",
        "          print(f'Tags: none\\n')      \n",
        "\n",
        "      # Increment the article count\n",
        "      count += 1\n",
        "\n",
        "      # Stop once articles reach the SHOW_TOP number\n",
        "      if count == show_top:\n",
        "        break"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 65,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "im9BmZYJ614y",
        "outputId": "9c54028f-c404-4111-ccfc-c603d62c45ac"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "------  You are reading...  ------\n",
            "[ID 70] Article: aragones angered by racism fine spain coach luis aragones is furious after being fined by the spanish football federation for his comments about thierry henry.  the 66-year-old criticised his 3000 euros (£2 060) punishment even though it was far below the maximum penalty.  i am not guilty  nor do i ...\n",
            "\n",
            "------  You might also like...  ------\n",
            "[ID 23] Article: ferguson urges henry punishment sir alex ferguson has called on the football association to punish arsenal s thierry henry for an incident involving gabriel heinze.  ferguson believes henry deliberately caught heinze on the head with his knee during united s controversial win. the united boss said i...\n",
            "Tags: sir alex ferguson, arsenal, thierry henry, gabriel heinze, football association, manchester united\n",
            "\n",
            "[ID 51] Article: mourinho defiant on chelsea form chelsea boss jose mourinho has insisted that sir alex ferguson and arsene wenger would swap places with him.  mourinho s side were knocked out of the fa cup by newcastle last sunday before seeing barcelona secure a 2-1 champions league first-leg lead in the nou camp....\n",
            "Tags: chelsea, jose mourinho, sir alex ferguson, arsene wenger, fa cup, soccer\n",
            "\n",
            "[ID 41] Article: mcleish ready for criticism rangers manager alex mcleish accepts he is going to be criticised after their disastrous uefa cup exit at the hands of auxerre at ibrox on wednesday.  mcleish told bbc radio five live:  we were in pole position to get through to the next stage but we blew it  we absolutel...\n",
            "Tags: alex mcleish, rangers, auxerre, uefa cup, bbc radio five live\n",
            "\n",
            "[ID 42] Article: premier league planning cole date the premier league is attempting to find a mutually convenient date to investigate allegations chelsea made an illegal approach for ashley cole.  both chelsea and arsenal will be asked to give evidence to a premier league commission  but no deadline has been put on ...\n",
            "Tags: premier league, chelsea, ashley cole, arsenal, date\n",
            "\n",
            "[ID 14] Article: ireland 21-19 argentina an injury-time dropped goal by ronan o gara stole victory for ireland from underneath the noses of argentina at lansdowne road on saturday.  o gara kicked all of ireland s points  with two dropped goals and five penalties  to give the home side a 100% record in their autumn i...\n",
            "Tags: ireland, argentina, ronan o gara, rugby, lansdowne road\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# Show the recommended articles\n",
        "get_recommendations(READING_IDX,similarity,SHOW_TOP)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IVhljS_4ILM2"
      },
      "source": [
        "Keeping to the Section 1.3 example, here we see how the classification and extraction steps have improved our recommendation outcome.\n",
        "\n",
        "First, now the politics article (ID 86) doesn't get recommended anymore. And now we have the tags related to each article being generated. \n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7_Mkm7c7_X5D"
      },
      "source": [
        "Let's try a couple of other articles in business and tech and see the output..."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TNQpkgXvkNtI"
      },
      "source": [
        "Business article (returning recommendations around German economy and economic growth/slump):"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 66,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "utbxE-k7_U1c",
        "outputId": "ca7b6085-6360-46a7-ed39-1c367e3317ef"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "------  You are reading...  ------\n",
            "[ID 1] Article: german business confidence slides german business confidence fell in february knocking hopes of a speedy recovery in europe s largest economy.  munich-based research institute ifo said that its confidence index fell to 95.5 in february from 97.5 in january  its first decline in three months. the stu...\n",
            "\n",
            "------  You might also like...  ------\n",
            "[ID 56] Article: borussia dortmund near bust german football club and former european champion borussia dortmund has warned it will go bankrupt if rescue talks with creditors fail.  the company s shares tumbled after it said it has  entered a life-threatening profitability and financial situation . borussia dortmund...\n",
            "Tags: german football club, borussia dortmund, bankruptcy, profitability and financial situation\n",
            "\n",
            "[ID 2] Article: bbc poll indicates economic gloom citizens in a majority of nations surveyed in a bbc world service poll believe the world economy is worsening.  most respondents also said their national economy was getting worse. but when asked about their own family s financial outlook  a majority in 14 countries...\n",
            "Tags: bbc, economy, poll, citizen, financial, outlook\n",
            "\n",
            "[ID 8] Article: car giant hit by mercedes slump a slump in profitability at luxury car maker mercedes has prompted a big drop in profits at parent daimlerchrysler.  the german-us carmaker saw fourth quarter operating profits fall to 785m euros ($1bn) from 2.4bn euros in 2003. mercedes-benz s woes - its profits slid...\n",
            "Tags: mercedes, daimlerchrysler, car profits, luxury car maker\n",
            "\n",
            "[ID 32] Article: china continues rapid growth china s economy has expanded by a breakneck 9.5% during 2004  faster than predicted and well above 2003 s 9.1%.  the news may mean more limits on investment and lending as beijing tries to take the economy off the boil. china has sucked in raw materials and energy to fee...\n",
            "Tags: china, economy, growth, raw materials, energy, investment, lending, Beijing\n",
            "\n",
            "[ID 96] Article: bmw to recall faulty diesel cars bmw is to recall all cars equipped with a faulty diesel fuel-injection pump supplied by parts maker robert bosch.  the faulty part does not represent a safety risk and the recall only affects pumps made in december and january. bmw said that it was too early to say h...\n",
            "Tags: bmw, diesel, robert bosch, recall\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# A business news article example (ID 40)\n",
        "\n",
        "# Enter your article ID\n",
        "READING_IDX = 1\n",
        "\n",
        "# Get embedding for the article\n",
        "reading = embeds[READING_IDX]\n",
        "\n",
        "# Get the similarity between the target and candidate articles\n",
        "similarity = get_similarity(reading,embeds)\n",
        "\n",
        "# Show the recommended articles\n",
        "get_recommendations(READING_IDX,similarity,SHOW_TOP)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VYitPtmJkUo5"
      },
      "source": [
        "Tech article (returning recommendations around consumer devices):"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 67,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XGpZxk0j_6G4",
        "outputId": "d287c789-52ac-4cfe-92e3-3b605570a876"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "------  You are reading...  ------\n",
            "[ID 71] Article: camera phones are  must-haves  four times more mobiles with cameras in them will be sold in europe by the end of 2004 than last year  says a report from analysts gartner.  globally  the number sold will reach 159 million  an increase of 104%. the report predicts that nearly 70% of all mobile phones ...\n",
            "\n",
            "------  You might also like...  ------\n",
            "[ID 3] Article: lifestyle  governs mobile choice  faster  better or funkier hardware alone is not going to help phone firms sell more handsets  research suggests.  instead  phone firms keen to get more out of their customers should not just be pushing the technology for its own sake. consumers are far more interest...\n",
            "Tags: mobile choice, phone firms, consumers, lifestyle\n",
            "\n",
            "[ID 69] Article: gates opens biggest gadget fair bill gates has opened the consumer electronics show (ces) in las vegas  saying that gadgets are working together more to help people manage multimedia content around the home and on the move.  mr gates made no announcement about the next generation xbox games console ...\n",
            "Tags: bill gates, consumer electronics show, gates, las vegas, multimedia content, next generation xbox games console\n",
            "\n",
            "[ID 46] Article: china  ripe  for media explosion asia is set to drive global media growth to 2008 and beyond  with china and india filling the two top spots  analysts have predicted.  japan  south korea and singapore will also be strong players  but china s demographics give it the edge  a media conference in londo...\n",
            "Tags: china, india, media, growth, japan, south korea, singapore\n",
            "\n",
            "[ID 19] Article: moving mobile improves golf swing a mobile phone that recognises and responds to movements has been launched in japan.  the motion-sensitive phone - officially titled the v603sh - was developed by sharp and launched by vodafone s japanese division. devised mainly for mobile gaming  users can also ac...\n",
            "Tags: sharp, vodafone, v603sh, mobile gaming, golf swing, motion-sensitive phone\n",
            "\n",
            "[ID 63] Article: what high-definition will do to dvds first it was the humble home video  then it was the dvd  and now hollywood is preparing for the next revolution in home entertainment - high-definition.  high-definition gives incredible  3d-like pictures and surround sound. the dvd disks and the gear to play the...\n",
            "Tags: high-definition, dvd, home entertainment\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# A tech news article example (ID 30)\n",
        "\n",
        "# Enter your article ID\n",
        "READING_IDX = 71\n",
        "\n",
        "# Get embedding for the article\n",
        "reading = embeds[READING_IDX]\n",
        "\n",
        "# Get the similarity between the target and candidate articles\n",
        "similarity = get_similarity(reading,embeds)\n",
        "\n",
        "# Show the recommended articles\n",
        "get_recommendations(READING_IDX,similarity,SHOW_TOP)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f55W2I9RZzXz"
      },
      "source": [
        "In conclusion, this demonstrates an example of how we can stack multiple NLP endpoints together to get an output much closer to our desired outcome.\n",
        "\n",
        "In practice, hosting and maintaining multiple models can turn quickly into a complex activity. But by leveraging Cohere endpoints, this task is reduced to a simple API call."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "Article Recommender with Text Embedding, Classification, and Extraction.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3.10.0 64-bit ('3.10.0')",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.4"
    },
    "vscode": {
      "interpreter": {
        "hash": "1fb8019e3560b882083e525615cf48e713d3a7345a15eb723d805e91aa410aac"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
